2a.1. How does the unit ensure that the assessment system collects information on candidate proficiencies outlined in the unit's conceptual framework, state standards, and professional standards?

The unit’s conceptual framework (Exhibit C1.5 ULM KSD Chart), which is the foundation of the Unit Assessment System (UAS), incorporates the state and professional standards (Exhibit C1.3 Alignment Matrix). The goal of the unit is to produce highly qualified learning facilitators who are prepared to enhance the learning of K-12 students. The UAS, developed in accordance with Louisiana and NCATE guidelines, provides an accountability blueprint for initial and advanced education candidates from program entry through completion of their chosen program as well as their performance in the K-12 schools after program completion (Exhibit 2a1.1 Unit Assessment Handbook). Designed to reflect the Unit’s Conceptual Framework (Exhibit C1.1 Conceptual Framework Description Document), and to carry out the University’s Mission (Exhibit A2.1 ULM Vision/Mission Statement) of creating exemplary educational programs, the UAS encompasses both professional and state standards in an accountability and data collection system that facilitates candidate and faculty understanding of unit and program expectations, while providing valuable data to inform unit, program, and course improvements.

The unit has worked diligently to build a firm foundation for the assessment system. Over the past several years, the unit has matured and become more sophisticated in understanding and determining what data are needed and valued for ongoing improvement. The unit understands the complexity of assessment at all levels and recognizes that a strong assessment system for this unit will require continued planning, development, implementation, and refinement.

The Assessment Committee oversees assessment within the unit and coordinates assessment practices among programs. The Assistant Dean of Assessment and Accreditation along with the Director of Research and Data, both full-time positions in the unit, collect and assist the faculty in analyzing the data (e.g., PRAXIS scores, key assessment scores for specialty program area reports, surveys) from various areas of the assessment system at the end of the each Spring semester. This data is then disseminated to program chairs as well as department heads.

The Assistant Dean, along with the Director of Research, maintains the electronic assessment system (TaskStream), which is used to collect candidate data. Systematic and routine checkups are performed every semester to ensure candidates submit the required assessment (e.g., signature assessments, surveys, portal reviews) and faculty and supervisors evaluate this assessment on time. The Assistant Dean presents proposed policy and practice changes to the Assessment Committee. These changes may be initiated at the program, department, or the committee level, such as Teacher Education Council, consisting of faculty from the College of Arts and Sciences and other professional educators. The initiating faculty member must: 1) complete change of signature assessment/program form; 2) provide rationale for the change; 3) collect quantitative or qualitative data; 4) have all faculty members and program chairs sign off; 5) present to Assistant Dean of Assessment and Accreditation. The Assistant Dean of Assessment and Accreditation then presents this information to the Assessment Committee during one of its regular meetings and the requesting faculty member is present for clarification purposes. After review by the committee, the faculty member is notified of the status of the change request, and if approved, implementation of change is actuated. If it is a program or unit change, the Dean’s Advisory Council and PK-16 Council are also notified during their next scheduled meeting. The initiating faculty member is notified if needed. If change is to a signature assessment on TaskStream the data from the old signature assessment are backed up and stored.

2a.2. What are the key assessments used by the unit and its programs to monitor and make decisions about candidate performance at transition points such as those listed in Table 6? Please complete Table 6 or upload your own table at Prompt 2a.6 below.

Unit transition points, or portals, are depicted in Exhibit 2a2.1 Unit Assessment Portal Requirement and are designed to provide a somewhat comparable structure across programs and levels. The key assessments used to monitor and make decisions about candidate performances at each portal are outlined in Table 6. As indicated, key assessments include course-specific signature assessments to evaluate candidate performance of particular KSDs as well as additional indices of proficiency.

Table 6
Unit Assessment System: Transition Point Assessments (see Exhibit 2a2.1 Unit Assessment Portal Requirement)

INITIAL

ADVANCED

DOCTORAL

2a.3. How is the unit assessment system evaluated? Who is involved and how?

The Assessment Committee, comprised of the Unit Dean, Associate Dean, Assistant Dean, Director of Field Experiences, Director of Research and Data, a graduate candidate, Technology Coordinator, and Curriculum and Instruction Department Head, all of whom are appointed by the Unit Dean, evaluate the assessment system as well as the kind of data generated. The Committee is supported by input from a variety of sources and holds regular meetings, both formally and informally, to monitor progress, evaluate the assessment system, and modifies the system when the need arises (Exhibit 2a3.1 Assessment Change Forms and Signatures). The committee informs stakeholders of results found and implemented changes. The Assessment Committee works in conjunction with regular NCATE meetings and participates in PK16+ Advisory Council meetings throughout the academic year to ensure stakeholder involvement and support for system decisions (Exhibit 2a3.2 Minutes of Assessment Committee Meetings). At the end of each year, the Assessment Committee reviews the system and makes changes. The changes made are then disseminated to department heads and faculty members to implement and pass on to students. Although some elements of the assessment system, such as basic program entry and exit requirements, have been in place for decades, others, such as mid-program portals and emphasis upon demonstrated candidate performance indicators used for comprehensive data analysis, have been instituted in more recent years.

As the University has moved through redesign during the past seven years, additional modifications of the assessment system have been implemented to more closely align candidate performances with NCATE, state, and program standards. All course signature assessment rubrics now clearly relate candidate performance to the knowledge, skills, and dispositions (KSDs) espoused by the ULM Conceptual Framework (Exhibit C1.2 Learning Facilitator Model Description). Presented as a matrix (Exhibit 2a3.3 Matrix of Courses), all course syllabi display the interconnectedness between course objectives, KSDs, course assessments, NCATE and state standards, Louisiana Components of Effective Teaching (LCET), program/professional standards, and expected mastery level for candidates. The Unit Assessment System is designed in such a way that candidates’ transition points, program and unit expectations, timelines, and responsibilities are clearly delineated (Exhibit 2a2.1 Unit Assessment Portal Requirements). Assessment data for the purpose of unit and program analyses are compiled at twelve identified portals that progress through the initial, advanced, and doctoral programs, along with follow-up data which are also collected for each program. Some of the data collected within these points, such as the entry requirements of ACT or GRE scores, high school or undergraduate GPA, and the inclusion of Signature Assessments of core courses, are common to all programs, but each program has assessments that are selected for their relevance to that particular program. (Exhibit 2a1.1 Unit Assessment Handbook).

2a.4. How does the unit ensure that its assessment procedures are fair, accurate, consistent, and free of bias?

To maintain consistency of interpretation of assessment data across the unit, the NCATE three-point rubric was adopted in the Fall of 2006 as the common statute for interpreting designated signature assessments and reporting the conceptual framework knowledge, skills, and dispositions (Exhibit C1.2 Learning Facilitator Model Description). The decision serves to clarify performance levels across the unit and facilitate interpretation of data. The rubric, which describes and differentiates three levels of observed cognitive and relational attainment, was chosen specifically because it aligns with NCATE standards (Exhibit 2a4.1 Three Point Scoring Rubric and Committee Approval Minutes).

Procedures for ensuring content validity have been established for signature assessment rubrics (Exhibit 2a4.2 Signature Assessment Guidelines). Validity and reliability of assessment instruments are established through the use of expert panels for content validity and workshops on how to score using a 3 point scale. (Exhibit 2a4.3 Assessment Validity and Reliability). Bias is avoided by use of multiple assessors, multiple instruments, and portal review committees (Exhibit 2a4.4 Undergraduate and Graduate Committee) who share decision-making concerning candidate performance.

Data from candidates, completers, faculty, and other members of the professional community are collected from multiple assessments and include both internal and external sources (Exhibit 2a4.5 List of CEHD committees). Program level assessments include course-specific activities in the form of Signature Assessments (Exhibit 2a4.2 Signature Assessment Guidelines). Unit level-Signature Assessment data are aggregated across programs. Standardized tests scores and Louisiana Teacher Assistance and Assessment Program (LaTAAP) provide the unit with data based upon consistent, reliable, and nationally validated criteria for candidate performance to be used in comparative analyses and assurance of candidates’ mastery of content.

Utilization of internal and external sources of data and evaluators, including both university faculty and school district personnel, serve to control for bias. Signature assessments are shared with candidates before they are used.

The final evaluation for student teachers and interns employs the previously validated LCET Standards (Exhibit 2a4.6 LCET Standards) that are also utilized by state evaluators in the two-year state licensure process. Scoring of candidates with this instrument is completed collaboratively by the school site and the university supervisors to ensure consensus. Portal portfolios are also evaluated through a consensus process by members of the Undergraduate and Initial Review Committee or the Graduate Review Committee (Exhibit 2a4.4 Undergraduate and Graduate Committee) Candidates are assured of multiple assessors evaluating assessments that are submitted for review at each portal.

2a.5. What assessments and evaluation measures are used to manage and improve the operations and programs of the unit?

Similar to how data is used for program management and improvement, multiple sets of survey data are also used for unit management and improvement such as: 1) Student Advisement Survey, 2) Graduation Exit Surveys (for all students in the unit), 3) Unit Operations Assessment Survey for Candidates, 4) Faculty Evaluation by both candidates and department heads, 5) Faculty and Administration Unit Evaluation Procedure Survey.

Faculty evaluations are completed by the department heads for each faculty member in their specific department area. A new faculty evaluation instruments is in the process of being developed, with base questions consistent and unified across the unit. The department heads have the choice to add questions to provide better questioning and observing of their faculty (Exhibit 2a5.1 Faculty, Unit, Exist Evaluation Instruments).

The Faculty and Administration Unit Evaluation Procedure Survey is given to unit faculty once a year for completion. The survey includes topics such as: how well the unit addresses objectives outlined in the mission statement and conceptual framework, how well the unit achieves a higher caliber of educational impact, how well the unit fulfills campus citizen obligations, and how prominent faculty have become in their teaching, research, and service contributions. (Exhibit 2a5.1 Faculty, Unit, Exist Evaluation Instruments)

All candidates must be advised prior to registration every semester and complete a survey to provide feedback for improving the advisement process. The unit designed an Advising Survey that is given to all students when they complete the advising process for the semester (Exhibit 2a5.1 Faculty, Unit, Exist Evaluation Instruments). Data from the advising form is used to help manage and to make improvements the advisement process.

Once a year, all teacher education candidates are asked to complete a Unit Operations Assessment Survey which is conducted through the unit. The survey includes topics such as: overall unit operations, assessment of performance by professors and supervisors, advisement experiences, management and organization of the learning experience/information needs, within-unit support, nature of interactions within the unit, operations outside the unit, admission procedures, and use of assessment system. Data from this survey is used to help manage the operations within the unit. (Exhibit 2a5.1 Faculty, Unit, Exist Evaluation Instruments)

All graduating teacher education candidates are given either an Undergraduate Exit Survey or a Graduate Exit Survey. Both surveys address the candidate’s experiences in taking courses and also resources available on campus to enhance their learning. The surveys also address how well the unit prepared them to be qualified teachers. Data collected from these surveys are used to make program and unit changes. (Exhibit 2a5.1 Faculty, Unit, Exist Evaluation Instruments)

2a.6. (Optional Upload for Online IR) Tables, figures, and a list of links to key exhibits related to the unit's assessment system may be attached here. [Because BOE members should be able to access many exhibits electronically, a limited number of attachments (0-5) should be uploaded.]